Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Administrator, Curator, or Staffbleblanc Date: Dec 14, 2002 5:06am
Forum: etree Subject: How Shows Travel through the System

Please read all the way through this. It's more than an answer to the question above, but hopefully it clears up some more questions. I believe that step 2 answers your question.

Archive contents work like this.

1. Folks like you with an urge to make this collection bigger and better find shows in their collection that are not present in the Archive and prepare them for upload. They set up the upload using the Arrange an Upload link. Clicking on the I'm Ready button automatically creates an upload folder on a server for you to use and gives you login info to get to it and begin transferring files.

2. Each of the servers run an md5 check (also a flac fingerprint check) script every day against the contents in the upload folder. The script begins at Midnight Pacific Standard Time and runs until it completes. If the server is heavily taxed for resources (usually this is the case as it passes 50% full), it tends to run the script all 24 hours without completing. When it reaches midnight again, it stops where it is and starts over (this is the reason why sometimes you can't find the results of your uploads. It didn't get to it when it started over)

Since all the servers are running on Unix, and Unix is case sensitive, some md5's that pass on Windows will show up as missing on the server. For example, if your files are named:

dmb2002-09-08d1T01.shn (note the T)

and the md5 has them named:

dmb2002-09-08d1t01.shn (note the t)

Then the results file will tell you that the file is missing. It is the uploaders responsibility to make sure the md5's are ready for a Unix system.

As the server checks the contents, it writes the results to a file on the server that can be opened with any standard text editing program (wordpad in windows) This can be found when you log in as an uploader, before double clicking on your upload folder. They are named by the day of the week. (i.e. md5results.uploads.Mon is the results file that ran on Monday)

3. Each day, the administrators get a copy of these results and, time permitting, import shows that both pass md5 or flac check, and have a valid info file. If the show is by an artist that hasn't approved of being included in the Archive( See The List), we put it away to be brought out in case they do approve.

So, this leads to a few things that make for a perfect contributor for the Archive.

Patience You've got to understand that this is an Archive, not just a collection of servers where we get shows out as fast as possible. Our goal is to Archive these shows for Future Generations, so we're trying pretty hard to keep the data as accurate as possible. Sometimes your upload will sit in the upload folder for 2-3 weeks before getting imported because the server is so taxed for resources that it can't get to checking your upload until we start using another server and the load is removed. 2-3 weeks is just a drop in the bucket compared to the decades that it will be available, right? We want to know that it passes, so we wait.

Also, understand that the admins have jobs and school like the rest of you. :)

Know the system. Read the FAQ's before posting to the board. Most questions are answered in there. If it isn't, kindly request that your answer be put in there so other folks will know.

Don't give up on it. This resource is nothing without hundreds of you folks adding stuff. Keep taping and sending in shows!

If you know the answer to a post, please respond. The admin's can't get to every question on their own. Lets free up their time so they can concentrate on the servers, bands, and website.

This post was modified by bleblanc57 on 2002-12-14 13:06:37

Reply to this post
Reply [edit]

Poster: twiley Date: Apr 21, 2003 2:43am
Forum: etree Subject: Re: How Shows Travel through the System

>Server: etree04.archive.org (port 21)
>Username: etree
>Password: archive
>Directory: uploadtwiley69@hotmail.com


Well I follow all the steps and get to this message and I am trying to upload Bernie Worrell and the WOO Worriors DSBD to this server and get this.....


250 OK. Current directory is /
PWD
257 "/" is your current location
Cached directory reloaded.
MKD Server: etree04.archive.org (port 21)
550 Sorry, anonymous users are not allowed to create directories


I also do not see my directory or can't create it anyone know????


Also the files are labeled correctly per Diana's suggestions for Bernie = woo2002-04-20.shnf

What do I do to actually get this to upload?

TIA,

Tim

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffDiana Hamilton Date: Apr 21, 2003 3:07am
Forum: etree Subject: Re: Uploading

Ah, looks like you need to log in to etree05, not etree04. Good luck! Jon has a current note on the msgboard about etree05 being funky, that will be a separate problem later, likely. :O

While I'm posting about uploading, useful urls are:
http://archive.org/contribute.php
with these 2 links on it:
http://archive.org/audio/etree-contribute.php?stage=upload_info
http://archive.org/audio/etree-contribute.php

This post was modified by hamilton on 2003-04-21 10:07:57

Reply to this post
Reply [edit]

Poster: bobZemuda Date: Dec 14, 2002 4:34am
Forum: etree Subject: Re: How Shows Travel through the System

So in theory, having an e-mail address that starts closer to the begining of the alphabet will be beneficial to getting your shows through the archive faster?

bb@xx.com would have a better chance of getting verified in comparison to
usagi@xx.com


Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffJonathan Aizen Date: Dec 14, 2002 6:21am
Forum: etree Subject: Re: How Shows Travel through the System

This will be hopefully be resolved within a month. I'm thinking of ways to construct the verification process so uploaders get automatic email notifications when their uploads fail, and so that the order of verification is random every night.

Using a bogus email address will only verify that we cannot get in touch with you should something go wrong.

I consciously make an attempt to work on the list backwards from time to time, excluding the beginning of the alphabet.

Regardless, I hope to implement a more intuitive and fair upload system when time permits.

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Staffmedlhuber Date: Dec 14, 2002 7:09am
Forum: etree Subject: Re: How Shows Travel through the System

I think an email notifcation would be great when an upload checks out. mainly because I notice, that when I send a show to the server, wheather it go to my upload directory or I import something into the mirror directory, i'll check the md5mirror file or md5upload file, and even though the show will checkout, it wont be moved out of mirror directory for days sometimes. AND, other times i'll check those md5miror or md5upload file...and they're incomplete(not my shns, the md5 file). For example, I just grabbed todays md5upload file....and the furthurest down the file output produced was

home/etree/uploadblane@canada.com

Now having my directory at uploadmedxxxx...how am I supposed to know how my uploads are doing? Am I doing something wrong in reading these md5check files? I just downloaded that file and opened it in notepad.....seems incomplete to me.

Personally, once a show passes md5...i'd like to see it moved out of the upload/mirror directories...doesnt even have to be imported into the rest of the site, just kinda move it outta uploads so the uploaders know it IS in fact complete and ready to go.

Sorry for the babbles, and i'm not trying to step on any toes either......guess i'm just a little confused and still new to archive.org....peace

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffJonathan Aizen Date: Dec 14, 2002 7:24am
Forum: etree Subject: Re: How Shows Travel through the System

You can always check yesterday's log too - that will have more results than today's.

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Staffbleblanc Date: Dec 15, 2002 12:44am
Forum: etree Subject: Re: How Shows Travel through the System

. AND, other times i'll check those md5miror or md5upload file...and they're incomplete(not my shns, the md5 file)

This was explained (in detail) above... Please go back and read it again. Simply put, the server has so much work to do that it doesn't get through the whole contents.

Sorry for the babbles, and i'm not trying to step on any toes either

There isn't a problem with asking questions, you should never have to apologize for it.

Personally, once a show passes md5...i'd like to see it moved out of the upload/mirror directories...

Your uploads aren't idle because we want them that way. They're idle because we want to make sure they check out before spreading them. That's more important than speed.

Moving them without generating the webpage detail link is not an option. In fact, the web page takes 20 seconds to fill out, it isn't an issue.

Sit back, burn a CD, and w-a-i-t. All passing shows get imported. Period.

I think by now, we're all aware that you folks are restless to add shows faster than fans can download them.

-Brad

This post was modified by bleblanc57 on 2002-12-15 08:44:53

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Staffbleblanc Date: Dec 14, 2002 5:06am
Forum: etree Subject: Re: How Shows Travel through the System

Yes. This is technically true. However, using a flubbed email address is a good way to have your uploads ignored for a while.

Does it really matter?

-Brad

This post was modified by bleblanc57 on 2002-12-14 13:06:05

Reply to this post
Reply [edit]

Poster: bobZemuda Date: Dec 14, 2002 4:49am
Forum: etree Subject: Re: How Shows Travel through the System

Well, actually, yes, it does matter to me. I got a new band added to the archive and I'm trying to get at least a show or two up and just wanted to make sure it was actually working properly because it has been a little while since I uploaded the first show.

I think that's fair.

As you sort of point out though, the archive is huge, and a great resource for live music enthusiasts.

Thanks for your help you did answer my question.

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Staffbleblanc Date: Dec 14, 2002 5:12am
Forum: etree Subject: Re: How Shows Travel through the System

You and everyone else, Bob (including me, I've got a People Like Us show that has been waiting for an md5check for 4 days now - and it will be the first show for them).

I think it's fair to tell you that nothing makes your situation unique - new band or not.

There's 1,454 complete live recordings of 74 artists here, try a few while you wait like the other 200 uploaders.

I'm sorry that it isn't sufficient for your liking, we're trying to make it work the best it can.

This post was modified by bleblanc57 on 2002-12-14 13:12:08

Reply to this post
Reply [edit]

Poster: Erich Date: Dec 14, 2002 5:15am
Forum: etree Subject: Re: How Shows Travel through the System

"Well, actually, yes, it does matter to me. I got a new band added to the archive and I'm trying to get at least a show or two up and just wanted to make sure it was actually working properly because it has been a little while since I uploaded the first show"

patience. try it. Ive been uploading tim reynolds shows for over a month now, and i dont mind waiting a week if i know that its going to be a perminant place to spread that music. its not a race.

Reply to this post
Reply [edit]

Poster: brooks Date: Dec 14, 2002 4:06pm
Forum: etree Subject: Re: How Shows Travel through the System

I think it was a totally fair question. No need for anyone to jump down the guy's back. There were some pretty angry replies earlier that have been apparently modified since they were orig. posted.
The archive is a very cool concept and I think we all really appreciate it. There have just been some questions about how it all works.
Let's all repeat the w00k mantra and be nice to each other : "It's all good"

Terms of Use (10 Mar 2001)